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MULTI-SCALE RETINEX FOR COLOR IMAGE ENHANCEMENT 



Zia-ur Rahman, Member IEEE 

Science and Technology Corporation 
101 Research Drive 
Hampton, VA 23666 

ABSTRACT 

The retinex is a human perception-baised image pro- 
cessing algorithnn which provides color constancy and 
dynamic range compression. We have previously re- 
ported on a single-scale retinex (SSR) and shown that it 
can either achieve color /lightness rendition or dynamic 
range compression, but not both simultaneously. We 
now present a multi-scale retinex (MSR) which over- 
comes this limitation for most scenes. Both color ren- 
dition and dynamic range compression are successfully 
accomplished except for some "pathological" scenes 
that have very strong spectral characteristics in a single 
band. 

1. INTRODUCTION 

A common problem with color imagery — digital or 
analog — is that of successful capture of the dynamic 
range and colors seen through the viewfinder onto the 
acquired image. More often than not, this image is a 
poor rendition of the actual observed scene. In 1986, 
Edwin Land presented the last version of his retinex[l] 
as a model for hunian color constancy. Hurlbert[2, 3] 
showed that there is no mathematical solution to the 
problein of removing lighting variations. Moore[4, 5] 
implemented a version of the retinex in analog VLSI for 
real-time dynamic range compression but encountered 
scene context dependent limitations and hence failed to 
achieve a generalized implementation. More recently 
we, inspired by the work of Land, Hurlbert, and Moore 
decided to delve into this commonly occurring, but 
surprisingly intractable, problem. Our initial research 
resulted in the single-scale retinex (SSR) that we have 
described in detail previously [6, 7, 8]. The SSR shows 
exceptional promise for dynamic range compression 
but does not provide good tonal rendition. In fact, a 
distinct trade-off controlled by the scale of the surround 
function exists between dynamic range compression 
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and tonal rendition, and one can be improved only at 
the cost of reducing the other. 

This paper describes our initial research in allevi- 
ating some of these trade-offs by using a multi-scale 
retinex (MSR), i.e. a retinex which combines several 
SSR outputs to produce a single output image which 
has both good dynamic range compression and color 
constancy, and good tonal rendition. The tonal rendi- 
tion, though, is still scene dependent to a certain ex- 
tent. We will briefly describe the MSR in Section 2. In 
section 3 we will provide some of the results of apply- 
ing the MSR to color images and compare our results 
with other techniques for image enhancement; Finally, 
in Section 4 we will discuss the future direction for this 
research. 

2. THE MULTI-SCALE RETINEX 

The MSR can be compactly written as 

N 
n=l 

{log [5i(x, y)] - log [Siix, y) * M„(a:, y)]) (1) 

where the subscripts i ERtGjB represent the three 
color bands, N is the number of scales being used, 
and Wn are the weighting factors for the scales. The 
Afn(x,y) are the surround functions given by 

Mn{x,y) = K„ejq>l-ix'+y')/<rl], 

where the ffn are the standard deviations of the Gaus- 
sian distribution that deteraiine the scale. The mag- 
nitude of the scale determines the type of information 
that the retinex provides: smaller scales providing more 
dynamic range compression, and larger scales provid- 
ing more color constancy. The Kn are selected so that 
J F{x, y)dxdy~ 1. Each of the expressions within the 
summation in Eq. 1 represents an SSR. 

The SSR has been previously defined [6] to have the 
following characteristics and properties: 
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1. The functional forni of the surround is a Gaus- 
sian. 

2. The placement of the log function is AFTER sur- 
roiind formation. 

3. The post-retinex signal processing is a "canon- 
ical" gain-offset rather than an automatic gain- 
offset. 

4. There is a trade-off between dynetmic range com- 
pression and tonal rendition which is governed by 
the Gaussian surround space constant. A space 
constant of 80 pixels was a reasonable compro- 
mise between dynamic range compression and 
rendition. 

5. A single scale seemed incapable of simultaneously 
providing sufficient dynamic range compression 
and tonal rendition. 

6. Violations of the gray- world assumption led to 
retinexed images which were either "grayed-out" 

; locally or globally or, more rarely, suffered from 
color distortion. 

The MSR combines the dynamic range compression 
of the small scale retinex with the tonal rendition of 
the large scale retinex to produce an output which 
encompasses both. 

As stated above, the MSR still suffers from graying- 
out of uniform zones much as the SSR did. The advan- 
tage that the MSR has over the SSR is in the com- 
bination of scales which provide both dynamic range 
compression and tonal rendition at the same time. The 
overall result of the application of the MSR is still more 
saturated than human observation, giving the final im- 
age a **washed-out" appearance, but it preserves most 
of the detail in. the 8cene^,^^h\s **graying" of areas 
of constant intensity occurs because the retinex pro- 
cessing enhances each color band as a function of its 
surround. The smaller values in the weaker channels 
get "pushedytip strongly, making them approximately 
equal in liiagnitude to the dominant channel, leading 
to a graying out of the overall region. Moore[4] en- 
countered this problem in his implementation of the 
retinex and atterripted to resolve it with using variable 
gains across the color channels. We do not attempt a 
solution in this paper but provide a detailed solution 
elsewhere. [9] However, the MSR produces a much bet- 
ter final image in terms of color, and dynamic range 
than the SSR. Figure 1 shows a comparison of the SSR 
and the MSR processing. The differences are easier to 
see in the original color images (see CD-ROM version 
of paper), but if one looks around the left side of the 



Figure 1: (a) Original (b) Single-scale Retinex (c) 
Multi-scale Retinex 
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face and in the area just above the right shoulder of 
the pictured man ■ one sees details for the MSR which 
are not evident in the SSR. Also the 'lialoing" artifacts 
peculiar to the SSR are eliminated in the MSR. 

3. RESULTS 

Figure 3 shows a comparison of the MSR with image 
enhancement niethods typically used for dynamic range 
compression. The scenes are selected to show the ef- 
fects of MSR processing on "good images" (top row), 
wide dynamic range compression that is achieved by 
the MSR (middle row), and color constancy (bottom 
row). Histogram equalization performs well for the 
child image, but begins to saturate in both the grass im- 
age and the cave image. The logarithmic non-linearity 
has the poorest performance for all three scenes, though 
its dynamic range compression capabilities are quite ev- 
ident in the grass scene. For the MSR processing, the 
uniform regions in the child scene tend to gray out, 
but the overall result is still quite good. For the grassy 
field, the MSR processing compresses the wide dynamic 
range well and brings out the colors in both the bright 
and the dark areas very well. For the cave image, the 
color of the inside rock, and the outside rock forma- 
tions are both brought out so they agree with actual 
observation. The CD-ROM version of the proceedings 
contains the color postscript figures and the compar- 
isons are much easier to make. 

The MSR output brings out most of the detail in the 
black regions but at the cost of enhancing the noise in 
these regions. This noise is a result of the poor signal- 
to-noise ratio in these areas. The traditional techniques 
are also able to enhance, the dark regions, but not to 
the same extent as the MSR. In fact, the MSR achieves 
a balance between enhancing the darks, yet; at the 
same time, retaining the colors in the bright regions, 
as opposed to traditional point hbn-linearities which 
tend to ienhance the darks at the cost of saturating the 
brights (Figs. 3(b,c)). Of course, the final rendition in 
still scene-dependent and can often be grayed-out if the 
origined scene contains large areas of constant intensity 
(Fig. 3(d)(top row)). 

The MSR output is different from existing tech- 
niques in that the overall effect of processing is scene 
dependent but the processing itself is not. In other 
words, though the overall effect adapts itself to the 
lighting vsiriations within the scene, the same process, 
with exactly the same control parameters <;an be used 
for any image. This is not true for other adaptive tech- 
niques since variations in lighting conditions imply vari- 
ations in the control parameters. 



4. FUTURE RESEARCH 

The main direction of further research, is to improve 
the color rendition of the MSR. Though it produces 
excellent dynamic range compression, the tonal rendi- 
tion is scene dependent and can be quite poor. Work 
is already underway on a newer version of the MSR 
which combines a post-filter with the MSR to produce 
an MSR which provides very good color rendition with 
a very slight loss in overall dynamic range compression. 
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